On the Accuracy of Compile-time Performance Prediction

In previous reports a compile-time prediction technique has been introduced that yields lowcost, highly parameterized performance models, to be used during the initial optimization loops in parallel system design. Aimed to provide an acceptable accuracy in the face of resource contention the approach is based on a combination of asymptotic bounding analysis and critical path analysis, yielding an analytical lower bound estimate. In this report we present an initial study into the accuracy of the analytical technique compared to the result from simulation in terms of both theory and experiments. For simple models it is conjectured that the maximum relative error of the lower bound estimate is typically independent of the system size and can be approximated by a simple expression. For complex models it is shown that the average relative deviation of the estimate is limited and predictable. A simple operational model parameter is proposed which provides a good indication of the credibility of the estimate. The validity of the overall technique is shown through a case study which involves dataow style execution of various statically mapped computations on a distributed-memory machine. This report concludes a series of reports which will form the basis for a thesis describing our approach to the performance modeling of parallel systems.